Touch drawing display apparatus and operation method thereof, image display apparatus allowing touch-input, and controller for the display apparatus

ABSTRACT

An electronic blackboard includes a display panel and a touch-panel, draws a line corresponding to a track formed by movement of the touched position on the displayed image, and erases an erroneously drawn line corresponding to a track specifying an operation other than drawing. In another aspect, an electronic blackboard includes a touch screen, a scroll unit for scrolling an image on the display screen when positions of multi-touch move in one same direction, and a page switching unit for detecting, after detection of multi-touch input, that a finger is moved away, and executing a process of returning the image on the display screen to a state before scrolling or a process of switching the image by one page in accordance with the moving direction of multi-touch positions, depending on whether the amount of movement of touch inputs is equal to or smaller than a threshold value.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a Divisional of copending U.S. application Ser. No.13/370,049, filed on Feb. 9, 2012, which claims priority under 35 U.S.C.§ 119(a) to Patent Applications No. 2011-027236 and No. 2011-027237filed in Japan on Feb. 10, 2011, all of which are hereby expresslyincorporated by reference into the present application.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to an image display apparatus having adisplay device and an input device integrated together, allowingtouch-input. More specifically, the present invention relates to a touchdrawing display apparatus that has functions of drawing operation andother operation regarding touch operations, and operation methodthereof, as well as to an image display apparatus switching images pageby page by touching and sliding images with a plurality of fingers andthe like and a controller for the display apparatus.

Description of the Background Art

As an image display apparatus allowing touch-input, a so-calledelectronic blackboard having a display device with a large screen,allowing drawing of an image or other processes upon detection oftouching of the display device by the user has been known. Theelectronic blackboard is useful when summarizing opinions ofparticipants or finding a preferable solution to a problem, for example,at a meeting. Electronic blackboards having various configurations havecome to be practically used, and one configured as a computer systemhaving a combination of a display device with a large display screen andan input device for detecting two-dimensional position coordinates suchas a touch-panel has been used.

Generally, an electronic blackboard apparatus successively reads piecesof information related to position coordinates designated by a pen or afinger and pieces of information related to amount of movement, anddisplays a track of inputs on the display device based on the readpieces of information. Consequently, the apparatus can realizeoperations as an electronic blackboard such as handwriting input.

Regarding the touch operation, Japanese Patent Laying-Open No. 6-175776(hereinafter denoted as '776 Reference) discloses a projection typepresentation device allowing drawing on a displayed screen image usingtouch operation. There is a problem that recognition of an originalimage becomes difficult when writing is repeated on the same screenimage. In order to solve this problem, in the device disclosed in thisreference, drawing is done while touch operation is continued in animage forming area, and when the touch operation in the image formingarea once ends and then a touch operation in the image forming area isdetected next, the former image is automatically deleted.

If a specific function other than drawing is allocated to a touchoperation different from a simple touch operation for drawing in anelectronic blackboard, while an image is being drawn in a drawing mode,it takes some time until a touch operation is determined to be the onefor executing the specific function. Typically in an electronicblackboard, drawing is done immediately after the touching, so as not tocause any time lag between the touch operation and drawing. Any time lagis stressful for the user. Such a configuration sometimes leads to anerroneous drawing not intended by the user. By way of example, it isoften the case that an operation of flicking with multi-touch (touchinga plurality of points simultaneously) is allocated to an operation ofscrolling screen images. In such a situation, if the user is drawingimages with single-touch operation (touching one point at a time) andthen makes a multi-touch flick operation for scrolling screen images, itis necessary to monitor the tracks of touched points for a prescribedtime period to determine that it is a flick operation. As the image isdrawn immediately in response to the touching to avoid any stress to theuser, it follows that an erroneous image not intended by the user havebeen drawn by the time when the operation is determined to be themulti-touch flick operation. While the image with the erroneous drawingdisappears from the screen image when the screen images are scrolled,drawing information containing the erroneous drawing is stored as it is.Therefore, when scrolled back to the original screen image, theerroneous drawing again appears on the screen image. When printing isdone, similarly, the erroneous drawing is also printed.

This problem is not limited to the electronic blackboard, and it occursin display devices allowing display of images drawn by touching, such astablet-type terminals. This problem cannot be solved by the techniquedisclosed in '776 Reference.

Advantages of an electronic blackboard include that it is possible todisplay or write (draw) images separately on a plurality of screenimages. Each unit of display of such images is referred to as a “page”as an analogy to a book.

In an electronic blackboard allowing display of a plurality of pages ofimages, how to switch pages easily is an issue. The easiest method maybe to provide a button for switching pages on the screen image. Thismethod, however, is problematic particularly when the screen image islarge, as it becomes difficult for the user to press the buttondepending on the position of the user.

In view of the foregoing, other than the method using the button, amethod of switching the screen images by touching the screen image witha finger and sliding the same is proposed. Here, if it is too simple,the operation would be the same as the usual drawing operation.Therefore, in order to distinguish the two different operations,Japanese Patent Laying-Open No. 11-102274 (hereinafter referred to as'274 Reference) proposes a device in which the screen image is switchedif the screen image is touched by a plurality of fingers and the like (aso-called multi-touch) and the plurality of touched positions are slidby more than a prescribed value in the same direction.

Japanese Patent Laying-Open Nos. 8-76926 (hereinafter referred to as'926 Reference) and 9-231004 (hereinafter referred to as '004 Reference)disclose methods as further development of the idea described above.According to the method described in '926 Reference, when an operationof touching and sliding is done, a page is turned, and the number ofpages turned at one time is changed depending on the number of touchingfingers (the number of touched positions). According to the methoddisclosed in '004 Reference, scrolling of images, turning of pages andmoving of cursors are executed in accordance with the movement andnumber of fingers touching the display screen surface.

According to the techniques disclosed in '274, '926 and '004 Referencesdescribed above, it is possible to turn pages by the simple operation oftouching and sliding. There is still a problem to be solved.Specifically, pages may be turned erroneously while normal input isbeing done. Particularly when input is being done using a finger, it ispossible that a finger other than the finger used for input happens totouch the screen surface and is detected as the multi-touch slide inputand, as a result, a page is turned unintentionally. Therefore, amechanism that can prevent unintended turning of a page even when suchan erroneous input is made has been desired. Further, a mechanism thatallows the user to easily understand what manner of input is necessaryto turn a page is also necessary.

SUMMARY OF THE INVENTION

In view of the problems described above, it is desired to provide atouch drawing display apparatus and operation method thereof that canmaintain correct drawing of images even when a touch operation otherthan the image drawing is made in the drawing mode.

Further, it is desired to provide an image display apparatus allowingtouch-input that allows the user to intuitively understand the method ofpage-by-page switching of screen images using multi-touch and that canprevent any trouble of display when an erroneous operation is made onsuch an occasion.

Further, it is desired to provide an image display apparatus allowingtouch-input that allows the user to intuitively understand the method ofpage-by-page switching of screen images using multi-touch, and of whichswitching operation is easy.

According to a first aspect, the present invention provides a touchdrawing display apparatus, including: a display unit that displays animage; a detecting unit that is arranged on the display unit and detectsa touched position; a drawing unit that draws, on an image displayed onthe display unit, a line corresponding to a track formed by movement ofthe detected position; and an erasing unit that erases, when anoperation other than drawing is specified by the track, a line drawnfrom a start point to an end point of the track specifying theoperation.

Preferably, the track for specifying the operation other than drawing isa track formed by a multi-touch operation of simultaneously touching aplurality of positions of the detecting unit.

According to a second aspect, the present invention provides a method ofoperating a touch drawing display apparatus that includes a display unitthat displays an image and a detecting unit that is arranged on thedisplay unit and detects a touched position, including the steps of:drawing, on an image displayed on the display unit, a line correspondingto a track formed by movement of the detected position; determiningwhether or not an operation other than drawing is specified by the trackwhile the drawing step is being executed; and erasing, if an operationother than drawing is specified by the track, a line drawn from a startpoint to an end point of the track specifying the operation.

As described above, according to the present invention, when a touchoperation other than that for drawing images is made in the drawingmode, the erroneous image drawn during the determination of the touchoperation other than that for drawing is not retained, and the drawnimage as intended by the user can be stored. Therefore, when the objectimage is displayed again, the erroneous image is not displayed and onlythe intended image is displayed.

According to a third aspect, the present invention provides an imagedisplay apparatus allowing touch-input, including: a touch detectingunit that has a display screen, displays page images page by page on thedisplay screen, and detects positions and the number of touch inputsthat designate positions on the display screen; a scroll unit thatscrolls, when a plurality of touch inputs are detected by the touchdetecting unit and their positions on the display screen move in onesame direction, an image displayed on the display screen along withmovement of positions of the plurality of touch inputs; and a first pageswitching unit that detects, after the plurality of touch inputs aredetected by the touch detecting unit, decrease in the number of touchinputs from an output of the touch detecting unit and, depending onwhether an amount of movement of the touch inputs is not larger than afirst threshold value, selectively executes a process of returning theimage on the display screen to a state before scrolling and a process ofswitching the image on the display screen by one page in accordance withthe direction of movement of the plurality of touch inputs.

When touching of the display screen surface with a plurality of fingers,that is, so-called multi-touch, is detected by the touch detecting unitand the touched positions move in the same direction, the scroll unitscrolls the screen image in that direction. Therefore, the user canintuitively understand that the screen image can be scrolled bymulti-touch sliding operation. On the other hand, if the amount ofmovement is not larger than the first threshold value and the number ofmulti-touching decreases after the multi-touch sliding is done, thescroll made by that time point is resumed to the original state beforescrolling. Therefore, even when the user erroneously touches the displayscreen surface with a plurality of fingers and slides, the originalscreen image can be resumed if the user becomes aware and moves his/herfingers away immediately. If the multi-touch sliding exceeding the firstthreshold value is done and then the user moves away his/her fingersfrom the screen, the screen image is switched by one page in accordancewith the direction of sliding. Thus, the pages of screen image can beswitched by the simple operation of multi-touch sliding.

As a result, an image display apparatus allowing touch-input that allowsthe user to intuitively understand the method of page-by-page switchingof screen images using multi-touch and that can prevent any trouble ofdisplay when an erroneous operation is made on such an occasion can beprovided.

According to a fourth aspect, the present invention provides acontroller for a display apparatus, used for the display apparatus thathas a display screen, displays an image received from outside on thedisplay screen and detects and outputs, to the outside, positions andthe number of touch inputs that designate positions on the displayscreen. The controller for the display apparatus includes: a scroll unitthat generates, when it is detected that a plurality of touch inputs aredetected by the display apparatus and their positions on the displayscreen move in one same direction, an image by scrolling the imagedisplayed on the display screen along with the movement of positions ofthe plurality of touch positions, and applies the generated image to thedisplay apparatus; and a page switching unit that detects, after theplurality of touch inputs are detected by the display apparatus,decrease in the number of touch inputs from an output of the displayapparatus, depending on whether an amount of movement of the touchinputs is not larger than a first threshold value, selectively executesa process of generating page image data for returning the image on thedisplay screen to a state before scrolling and a process of generatingpage image data for switching the image on the display screen by onepage in accordance with the direction of movement of the plurality oftouch inputs, and transmits the generated page image data to the displayapparatus.

According to a fifth aspect, the present invention provides an imagedisplay apparatus allowing touch input, including: a touch detectingunit that has a display screen, displays page images page by page on thedisplay screen, and detects positions and the number of touch inputsthat designate positions on the display screen; a scroll unit thatscrolls, when a plurality of touch inputs are detected by the touchdetecting unit and their positions on the display screen move in onesame direction, an image displayed on the display screen along withmovement of positions of the plurality of touch inputs; a page switchingunit that switches, in response to an amount of movement of theplurality of touch inputs exceeding a threshold value during scrollingof the image by the scroll unit, the image on the display screen by onepage in accordance with the direction of movement of the plurality oftouch inputs; and a returning unit that detects, from an output of thetouch detecting unit, decrease of the number of plurality of touchinputs before page switching by the page switching unit during scrollingof the image by the scroll unit, and returns scrolling of the image bythe scroll unit to a state before scrolling.

The plurality of touched positions are detected by the touch detectingunit, and when these positions move in the same direction, the scrollunit moves the screen image along the direction of movement of thetouched positions. Therefore, the user can intuitively understand thatpage switching is possible by the so-called multi-touch. Further, if theplurality of touched positions move in the same direction and thethreshold value is exceeded, the display of screen image is switched inaccordance with the direction of movement of touched positions in thecourse of scrolling. The screen image can be switched by the simpleoperation of so-called multi-touch. Further, for this purpose, what isnecessary is simply to move the touched positions continuously withoutmoving the fingers and the like away from the screen surface. Further,if the number of touch inputs decreases before page switching takesplace, the screen image returns to the state before the scrolling and,therefore, the original state can easily be resumed even when anerroneous operation is made.

As a result, an image display apparatus allowing touch-input that allowsthe user to easily understand that page-by-page switching of screenimages is possible by multi-touch input and allows easy switching ofpage-by-page switching of screen images can be provided.

According to a sixth aspect, the present invention provides a controllerfor a display apparatus, used for the display apparatus that has adisplay screen, displays an image received from outside on the displayscreen and detects and outputs, to the outside, positions and the numberof touch inputs that designate a position on the display screen. Thecontroller includes: a scroll unit that generates, when it is detectedthat a plurality of touch inputs are detected by the display apparatusand their positions on the display screen move in one same direction, animage by scrolling the image displayed on the display screen along withthe movement of positions of the plurality of touch positions, andapplies the generated image to the display apparatus; a page switchingunit that switches, in response to an amount of movement of theplurality of touch inputs exceeding a threshold value during scrollingof the image by the scroll unit, the image on the display screen by onepage in accordance with the direction of movement of the plurality oftouch inputs; and a returning unit that detects, from an output of thedisplay apparatus, decrease of the number of plurality of touch inputsbefore page switching by the page switching unit during scrolling of theimage by the scroll unit, and returns scrolling of the image by thescroll unit to a state before scrolling.

As described above, according to the present invention, the user canintuitively understand that page-by-page switching of screen images canbe done by using multi-touch, and any trouble in display can beprevented even when an erroneous operation is made on such an occasion.Further, by a simple operation of multi-touch sliding, page of imagescan be switched by one page. It is unnecessary to stop the multi-touchsliding operation.

The foregoing and other objects, features, aspects and advantages of thepresent invention will become more apparent from the following detaileddescription of the present invention when taken in conjunction with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a schematic configuration of anelectronic blackboard in accordance with an embodiment of the presentinvention.

FIG. 2 shows an example of a method of detecting a touch-input.

FIG. 3 shows an example of a displayed screen image.

FIG. 4 is a flowchart representing a control structure of a programrealizing erasure of an erroneous drawing in the electronic blackboardapparatus in accordance with a first embodiment of the presentinvention.

FIG. 5 shows coordinate data structure stored when a multi-touchoperation is carried out in accordance with the first embodiment of thepresent invention.

FIG. 6 shows a scroll operation realized by the multi-touch operation inaccordance with the first embodiment of the present invention.

FIG. 7 shows a scroll of a screen image in accordance with the firstembodiment of the present invention.

FIG. 8 is a flowchart representing a control structure of a programrealizing page switching by a multi-touch, in the electronic blackboardapparatus in accordance with a second embodiment of the presentinvention.

FIG. 9 shows a multi-touch sliding operation in accordance with thesecond embodiment of the present invention.

FIG. 10 shows an operation of sliding by a distance D₁ with multi-touchand then moving the fingers away, in accordance with the secondembodiment of the present invention.

FIG. 11 shows an operation of sliding by a distance D₂ longer than D₁with multi-touch and then moving the fingers away, in accordance withthe second embodiment of the present invention.

FIG. 12 shows a screen image after a page switch, displayed when anoperation of sliding by a distance D₂ and then moving the fingers awayis done, in accordance with the second embodiment of the presentinvention.

FIG. 13 shows a screen image displayed when an operation of sliding by adistance D₃ with multi-touch is done, in accordance with the secondembodiment of the present invention.

FIG. 14 shows a screen image displayed when the sliding operation iscontinued even after the screen image is switched by one page from thescreen image shown in FIG. 13.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

In the following description and in the drawings, the same componentsare denoted by the same reference characters. Their names and functionsare also the same. Therefore, detailed description thereof will not berepeated.

In the following, the term “touch” means that a position is madedetectable by an input position detecting device, and “touch” mayinclude touching and pressing the detecting device, just touching andnot pressing the detecting device, and coming very close to but nottouching the detecting device. As will be described later, a contacttype as well as a non-contact device may be used as the input positiondetecting device. Where the non-contact type detection device is used,“touch” means coming very close to the detecting device, that is, to adistance that allows detection of the input position.

First Embodiment

[Configuration]

Referring to FIG. 1, an electronic blackboard apparatus 100 inaccordance with the first embodiment of the present invention includes:a central processing unit (hereinafter denoted as CPU) 102; a read onlymemory (hereinafter denoted as ROM) 104; a random access memory(hereinafter denoted as RAM) 106; a storage unit 108; an interface unit(hereinafter denoted as IF unit) 110; a touch detecting unit 112; adisplay unit 114; a display control unit 116; a video memory(hereinafter denoted as VRAM) 118; and a bus 120. CPU 102 is for overallcontrol of electronic blackboard apparatus 100.

ROM 104 is a non-volatile storage storing programs and data necessaryfor controlling operations of electronic blackboard apparatus 100. RAM106 is a volatile storage. Storage unit 108 is a non-volatile storageretaining data even when power conduction is shut off, and implemented,for example, by a hard disk drive, a flash memory or the like. Storageunit 108 may be configured as a detachable unit. CPU 102 reads a programfrom ROM 104 to RAM 106 through bus 120 and executes the program using apart of RAM 106 as a work area. CPU 102 controls various units andcomponents of electronic blackboard apparatus 100 in accordance with aprogram or programs stored in ROM 104.

To bus 120, CPU 102, ROM 104, RAM 106, storage unit 108, IF unit 110,touch detecting unit 112, display control unit 116 and VRAM 118 areconnected. Data (including control information) is exchanged betweeneach of the units through bus 120.

IF unit 110 is for establishing connection with the outside such as anetwork, and transmits/receives image data to and from a computer or thelike connected to the network. Image data received from the outsidethrough IF unit 110 is recorded in storage unit 108.

Display unit 114 is a display panel (such as a liquid crystal panel) fordisplaying images. Display control unit 116 is provided with a drivingunit for driving display unit 114. Display control unit 116 reads imagedata stored in VRAM 118 at prescribed timing, generates signals fordisplaying as an image on display unit 114, and outputs the generatedsignals to display unit 114. The image data to be displayed is read byCPU 102 from storage unit 108 and transmitted to VRAM 118.

Touch detecting unit 112 is, for example, a touch-panel, detecting atouch operation by a user. Here, if there is a plurality of positionstouched by the user, touch detecting unit 112 successively outputscoordinates of each of the positions. As a result, by monitoring theoutputs of touch detecting unit 112, it is possible to know the numberof touched positions and their coordinates successively. Detection oftouch operation when a touch-panel is used for touch detecting unit 112will be described with reference to FIG. 2.

FIG. 2 shows an infrared scanning type touch-panel (touch detecting unit112). The touch-panel has arrays of light emitting diodes (hereinafterdenoted as LED arrays) 200 and 202 arranged in a line on adjacent twosides of a rectangular writing surface, respectively, and two arrays ofphotodiodes (hereinafter referred to as PD arrays) 210 and 212 arrangedin a line opposite to LED arrays 200 and 202, respectively. Infraredrays are emitted from each LED of LED arrays 200 and 202, and theinfrared rays are detected by each PD of opposite PD arrays 210 and 212.In FIG. 2, infrared rays output from LEDs of LED arrays 200 and 202 arerepresented by arrows.

The touch-panel includes, for example, a micro computer (a deviceincluding a CPU, a memory and an input/output circuit), and controlsemission of each LED. Each PD outputs a voltage corresponding to theintensity of received light. The output voltage from the PD is amplifiedby an amplifier. Since signals are output simultaneously from theplurality of PDs of PD arrays 210 and 212, the output signals are oncesaved in a buffer and then output as serial signals in accordance withthe order of arrangement of PDs, and transmitted to the micro computer.The order of serial signals output from PD array 210 represents the Xcoordinate. The order of serial signals output from PD array 212represents the Y coordinate.

When a user touches a point on the touch-panel with a touch pen 220, theinfrared ray is intercepted by the tip of touch pen 220. Therefore, theoutput voltage of PD that has been receiving the infrared ray before theinterception drops. Since the signal portion from the PD thatcorresponds to the touched position (XY coordinates) decreases, themicro computer detects a portion where the signal levels of received twoserial signals decreased, and thereby finds the position coordinates ofthe touched position. The micro computer transmits the determinedposition coordinates to CPU 102. The process for detecting the touchedposition is repeated periodically at prescribed detection interval and,therefore, if one point is kept touched for a time period longer thanthe detection interval, it follows that the same coordinate data isoutput repeatedly. The touched position can be detected in the similarmanner when the user touches touch detecting unit 112 with his/herfinger without using touch pen 220.

The technique for detecting the touched position described above is wellknown and, therefore, further description will not be given here. Atouch panel other than the infrared scanning type panel (such as acapacitive type, surface acoustic wave type or resistive type touchpanel) may be used as touch detecting unit 112. When a capacitive touchpanel is used, a position can be detected even when a finger or the likeis not actually touching (non-contact), if it is close enough to thesensor.

On display unit 114, a screen image such as shown in FIG. 3 isdisplayed. The display screen image of display unit 114 is divided to adrawing area 230 and a function button area 240. Drawing area 230 is forthe user to draw an image by touching operations. Specifically, XYcoordinates of touched position and track of its movement aretransmitted from touch detecting unit 112 to CPU 102 as described above.In accordance with the received coordinate data, CPU 102 writes aprescribed value in a corresponding memory address on VRAM 118. Whilepixel values of image data on VRAM 118 may be changed, here, it isassumed that VRAM 118 is provided with an area (hereinafter alsoreferred to as an overlay area) for storing drawing data, separate fromthe area for storing image data. Assuming that data of a memory addresswhere drawing is not done of the overlay area is “0”, CPU 102 writes “1”to a memory address that corresponds to the drawn position. Displaycontrol unit 116 superimposes and displays on display unit 114 the imagedata with the drawing data (the data in the overlay area). Specifically,on a point where the drawing data exists (the pixel having “1” recordedin the overlay area), the drawing data (preset color) is displayed, andon a point where the drawing data does not exist (the pixel having “0”recorded in the overlay area), the image data is displayed.

On function button area 240, function buttons 242 each having specificfunction allocated thereto are displayed. At a lower portion of drawingarea 230, a page operation area 250 is displayed. On this area, a NEXTbutton 252, a PREVIOUS button 254 and a page number indication box 256are displayed. When touched, NEXT button 252 feeds the displayed page(image data) to the right and shows the next page. When touched,PREVIOUS button 254 feeds the displayed page to the left and shows theprevious page. Page number indication box indicates the page number ofthe currently displayed page, of the plurality of pages as the object ofdisplay. The position of page operation area 250 is fixed and it doesnot move even during scrolling. By way of example, the data fordisplaying page operation area 250 on display unit 114 may be stored inan overlay area separate from the overlay area for drawing.

When the user touches NEXT button 252, coordinate data of the touchedposition is transmitted from touch detecting unit 112 to CPU 102. CPU102 determines that the received coordinate data represents a positionin the area where NEXT button 252 is displayed. As a result, CPU 102transmits image data of the next page to VRAM 118, and transmits a pagefeed command to display control unit 116. In response, display controlunit 116 generates a signal for displaying an image in the course ofpage feed from the image data corresponding to the currently displayedpage and the image data corresponding to the next page on VRAM 118, andoutputs it to display unit 114. Consequently, an image in the course ofpage feed appears on display unit 114. It is noted that only a frame ofthe currently displayed image may be moved on the display screen imageof display unit 114 in the course of page feed.

Functions allocated to function buttons 242 may include: a function ofdrawing by a touch operation (allowing selection of types of pens fordrawing); a function of opening a file (image data) saved in storageunit 108; an erasure function of deleting a drawing in a prescribedarea; a function of saving displayed image data in storage unit 108; anda function of printing displayed image data. Each function button 242 isdisplayed as an icon.

In the following, a method of operating electronic blackboard apparatus100, particularly a control structure of a program realizing the processfor not leaving any erroneous drawing when a function other than drawingis allocated to a special touch operation, will be described. In thefollowing description, it is assumed that the function button fordrawing has been pressed in the screen image shown in FIG. 3 andelectronic blackboard apparatus 100 is in a mode that drawing startswhen drawing area 230 of touch detecting unit 112 is touched. Further,it is assumed that the special touch operation is a flick operation tothe right/left with multi-touch at two or more points. Scroll to theright is allocated to a right-flick operation with multi-touch, andscroll to the left is allocated to a left-flick operation withmulti-touch. It is further assumed that no process other than drawing isallocated to any other multi-touch operation.

At step 300, CPU 102 determines whether or not touch detecting unit 112is touched. As described above, CPU 102 determines whether or notcoordinate data is received from touch detecting unit 112. Positioncoordinates (X coordinate, Y coordinate) of the touched point are outputin the order of touching, from touch detecting unit 112. If it isdetermined that there is a touch, the control proceeds to step 302.Otherwise, the control proceeds to step 306.

At step 302, CPU 102 stores coordinate data that have been received fora prescribed time period (coordinate data of touched points) in RAM 106in a manner that represents the order of reception. At step 306, CPU 102determines whether or not the program is to be ended. When an end buttonas one of the function buttons 242 is pressed, CPU 102 ends the program.Otherwise, the control returns to step 300, and CPU 102 waits for atouch.

At step 304, CPU 102 determines whether or not the touch detected atstep 302 is a multi-touch (whether a plurality of points are touchedsimultaneously). Specifically, CPU 102 calculates, among the pluralityof coordinate data stored at step 302, the distance between the firstreceived coordinate data and the coordinate data received next. If thedistance is a prescribed value or larger, CPU 102 determines that it isa multi-touch, and if the distance is smaller than the prescribed value,determines that it is a single touch.

The touch detection interval (period) is short and touching to eachpoint involves slight time difference and, therefore, even in the caseof multi-touch, coordinate data are output in order of touching fromtouch detecting unit 112. Therefore, by calculating the distance betweenthe coordinate data received first and each of some coordinate datareceived second and thereafter, it is possible to determine whether thetouch was a single touch or a multi-touch. The number of points touchedsimultaneously in the multi-touch operation is not limited to two andmay be three or more. Therefore, distances are calculated for aplurality of continuous points. The distance used as a reference fordetermining a multi-touch may be set appropriately. What is necessary isthat the distance as the reference is larger than the normal distancethe user moves the touched point in the detection interval. If it isdetermined to be a multi-touch, the control proceeds to step 320. If itis determined not to be a multi-touch, the control proceeds to step 310.

At step 310, CPU 102 draws an image in accordance with the track ofmovement of the touched point, on the image displayed on display unit114. Specifically, CPU 102 reads the plurality of coordinate data storedat step 302 from RAM 106, and writes prescribed data on correspondingpoints in the overlay area of VRAM 118 and on points on lines connectingthese points in order. Consequently, the image data and the drawing data(data in the overlay area) are combined and displayed on display unit114, as if drawing is done on the image. Since the process at step 312is repeated as described later, the overlay area is overwritten withexisting drawing data left as it is.

At step 312, CPU 102 temporarily stores the contents of drawing in RAM106. What is required is that the drawing can be reproduced and, hence,the method of storage may appropriately be selected. By way of example,the coordinate data stored in RAM 106 at step 302 with the ordermaintained may be retained as they are. Alternatively, the data in theoverlay area of VRAM 118 may be stored as two-dimensional image data inRAM 106.

At step 314, CPU 102 determines whether or not the touch is maintained.Specifically, CPU 102 determines whether or not coordinate data arecontinuously received from touch detecting unit 112. For instance, iftouch pen 220 or the user's finger moves away, reception of coordinatedata from touch detecting unit 112 stops. If it is determined that thetouch is maintained, control returns to step 302, and the processfollowing step 302 is repeated. If it is not determined that the touchis maintained, the control returns to step 300 and CPU 102 again waitsfor a next touch.

At step 320, CPU 102 re-stores the coordinate data that have been storedat step 302 in RAM 106 as coordinate data corresponding to eachmulti-touch position, that is, for each track, in RAM 106. By theprocess of step 304, for each of the plurality of points touchedsimultaneously, the coordinate data of first touched point (hereinafteralso referred to as “start point”) have been determined. Therefore, itis possible to determine which of the plurality of coordinate data otherthan the start point stored in RAM 106 corresponds to the track of whichof the plurality of start points, based on the distance from the startpoint. In accordance with the result of determination, the coordinatedata of the start point and the coordinate data of points on the trackof movement of the start point are stored as sequence data, in the orderof detection. Assume, for example, that two points were touchedsimultaneously. At step 302, sequence data 400 shown on the left side ofFIG. 5 are stored in RAM 106. Of these, (x0, y0) and (x1, y1) areassumed to be the coordinates of two points that were touched first. Thepoint (x0, y0) was touched little earlier than the point (x1, y1). Here,two data sequences having coordinate data 410 of the first start pointand coordinate data 420 of the second start point as heads,respectively, such as shown on the right side of FIG. 5, are stored inRAM 106. Coordinate data 410 of the start point and the following datasequence 412 represent one track. Coordinate data 420 of the start pointand the following data sequence 422 represent another track. The mannerof storage is similar if the number of multi-touch points is three ormore.

At step 322, CPU 102 draws an image on the image displayed on displayunit 114, in accordance with the track of touched points as at step 310.CPU 102 draws using the coordinate data re-stored in RAM 106 as shown,for example, in FIG. 5. All tracks of simultaneously touched points maybe drawn, or a track or tracks of only one or some of the points (forexample, a track of the earliest-touched point) may be drawn. FIG. 6shows an example in which drawings 500 are made and thereafter, a user510 flicked to the right while multi-touching with two fingers. By thisoperation, erroneous drawing 502 is made on the image. In FIG. 6, only adrawing corresponding to one of the two tracks is displayed. If step 312has already been executed and drawing with single touch has been made,the contents of drawing by the multi-touch is added to the drawncontents.

At steps 324, CPU 102 temporarily stores the contents drawn at step 322in RAM 106. The specific method is the same as that of step 312. If thecoordinate data re-stored at step 320 is to be retained as it is,different from step 312, only the coordinate data representing the trackdrawn at step 322 may be retained.

At step 326, CPU 102 determines whether the multi-touch is maintained.Specifically, CPU 102 determines whether or not a plurality ofcoordinate data are continuously received from touch detecting unit 112and whether these coordinate data correspond to the tracks of aplurality of start points determined at step 320. By way of example,assume that the user first touched with two fingers and then moved onefinger away. Then, the track that has been made by the moved finger islost, and the coordinate data received from touch detecting unit 112come to be only the coordinate data representing one track. Consideringa situation that three or more points are touched at first, it isdetermined that multi-touch is maintained if simultaneous touching of atleast two points is continued. If it is determined that multi-touch ismaintained, the control proceeds to step 328. If it is not determinedthat multi-touch is maintained, the control proceeds to step 314, andCPU 102 determines whether or not single touch is maintained, asdescribed above.

At step 328, CPU 102 determines whether or not the detected multi-touchoperation is an operation allocated to scrolling (an operationdesignating a scroll). Specifically, for each sequence of coordinatedata corresponding to the tracks of the plurality of touch points storedat step 320, CPU 102 determines a vector from the coordinate data of thestart point to the coordinate data of the last point, and determineswhether the vector is of a prescribed length or longer and whether thevector is in the positive direction along the X axis (whether Xcomponent is positive). If the vector is in the positive direction alongthe X axis, the detected multi-touch operation is determined to be anoperation allocated to a scroll to the right. If the vector is of aprescribed length or longer and in the negative direction along the Xaxis (X component is negative), the detected multi-touch operation isdetermined to be an operation allocated to a scroll to the left. If itis determined to be an operation allocated to a scroll, the controlproceeds to step 330. Otherwise, the control returns to step 320. Theprocess following step 320 is repeated until the vector reaches theprescribed length.

At step 330, CPU 102 deletes the coordinate data representing the trackof touched point stored in RAM 106 at step 324. Specifically, CPU 102deletes the drawing data written after the determination of multi-touch(for example, write data “0”), while maintaining the drawing data (forexample, drawing data drawn with single touch) written before thedetermination of multi-touch in the overlay area. Therefore, on thescreen image of display unit 114, the erroneous line drawn by themulti-touch operation is erased. FIG. 7 shows a displayed screen imageon display unit 114 during a scroll. An arrow 520 represents a scroll tothe right. In FIG. 7, drawing 500 formed by the user and displayed inFIG. 6 is maintained, while erroneous drawing 502 is erased. Since FIG.7 shows a state in which scroll to the right has already been done tosome extent, part of the drawing on the upper right portion is notshown.

At step 332, CPU 102 executes the right or left scroll, in accordancewith the result of determination at step 328. Before executing thescroll, CPU 102 saves the contents of drawing (with the erroneousdrawing erased) that have been temporarily stored in RAM 106 at steps312 and 324 in storage unit 108, and the control returns to step 300.Since the contents drawn in the overlay area are saved in storage unit108, the image with drawing 500 intended by the user can again bedisplayed if a left scroll is designated later.

By the above-described process, in electronic blackboard apparatus 100in accordance with the first embodiment, when a left or right flickoperation is done with multi-touch by a user, the erroneous line drawnby the multi-touch before the start of scrolling is not left and onlythe drawing intended by the user is saved, and then the scroll can beexecuted.

Though an example in which the image read from storage unit 108 isdisplayed as a background has been described above, it is not limiting.The background may be a uniform color image (for example, white, blackor gray), provided that a function of drawing in response to a touchoperation on touch detecting unit 112 is realized.

Though an example in which the special touch operation represents aleft/right scroll operation, that is, an example in which the scrollfunction is allocated to the special touch operation, has been describedabove, it is not limiting. The function to be allocated may be anyoperation other than drawing and it may be an upward/downward scroll, ascroll to a diagonal direction, or a page switch without scroll.Further, the function to be allocated may be a function (operation)other than drawing, allocated to a function button. If an operationallocated to a scroll in a direction other than the left/right directionis to be detected, the scrolling direction may be determined consideringnot only the X component but also the Y component of the vector.

Further, it is possible that a plurality of users touch electronicblackboard apparatus 100, each with a touch pen or a finger, and to drawa plurality of images simultaneously. In order to distinguish suchdrawing operations from the operation allocated to the scroll describedabove (the operation of left/right flick with multi-touch), what isnecessary is to determine whether the distance of points touchedsimultaneously is about the distance between one's fingers (for example,at most a few centimeters).

As to the operation to which a function other than drawing is allocated,it is not limited to a multi-touch operation, and it may be asingle-touch operation provided that the operation can be distinguishedfrom the normally conducted touch operation for drawing. By way ofexample, if a track from the start point to the end point of asingle-touch operation (a figure drawn continuously without liftingone's finger or pen) is a prescribed shape, the allocated operationother than drawing may be executed.

Though an example in which CPU 102 determines whether or not anoperation is a multi-touch operation has been described above, it is notlimiting. By way of example, a micro computer in touch detecting unit112 may determine whether or not an operation is a multi-touchoperation, and may transmit the result to CPU 102.

Though an electronic blackboard apparatus has been described above, itis not limiting, and the present invention is generally applicable todisplay apparatuses that allow drawing and operation of screen images bytouching, including tablet type terminal devices.

Second Embodiment

[Configuration]

An electronic blackboard apparatus in accordance with the secondembodiment of the present invention has the same configuration as thatshown in FIG. 1 representing the electronic blackboard apparatus inaccordance with the first embodiment. The method of detecting a touchinput and the displayed screen images of the electronic blackboardapparatus in accordance with the second embodiment are also the same asthose described with reference to FIGS. 2 and 3 regarding the electronicblackboard apparatus in accordance with the first embodiment. Therefore,in the following, the electronic blackboard apparatus in accordance withthe second embodiment will be described as “electronic blackboardapparatus 100”, referring to FIGS. 1 to 3 where appropriate.

A program realizing the page switch function of electronic blackboardapparatus 100 in accordance with the second embodiment using thehardware shown in FIG. 1 when executed by CPU 102 shown in FIG. 1 hasthe following control structure.

Referring to FIG. 8, the program includes steps 600, 602 and 604. Atstep 600, CPU 102 monitors an output of touch detecting unit 112, and inresponse to an output detecting a touch by the user from touch detectingunit 112, the control flow proceeds to the next step. At step 602, CPU102 again detects a new position of touching by a finger based on anoutput from touch detecting unit 112, and the control proceeds to thenext step. At step 604, based on the results from steps 600 and 602, CPU102 determines whether or not the touch by the user is a multi-touchmade with two or more fingers. Touch detecting unit 112 has a functionof outputting coordinate data whose number corresponds to the number oftouching, in response to the touching by the user. Therefore, thedetermination as described above can be made based on the outputs fromtouch detecting unit 112.

The program further includes a step 606. At step 606, CPU 102 stores, ifthe determination at step 604 is positive, the touch positions detectedat steps 600 and 602 in RAM 106 (see FIG. 1). Since a multi-touch isdetected, here, coordinate values corresponding in number to the numberof detected touches are stored. In the subsequent process, thecoordinate data of each finger are detected for every detected touchposition, and the series of data are stored as a sequence in RAM 106.

The program further includes steps 608, 610 and 630. At step 608,following step 606, CPU 102 detects the present positions of fingersfrom the outputs of touch detecting unit 112. At step 610, based on theresult of detection at step 608, CPU 102 determines whether any one ofthe multi-touch detection outputs is lost (whether or not any finger hasbeen moved away from touch detecting unit 112). If the determination atstep 610 is positive, at step 630, CPU 102 compares the X coordinate ofthe last detected finger position with the X coordinate of the firstfinger position stored at step 606, and determines whether or not anabsolute value of difference between the two coordinates is larger thana threshold value D_(TH1). In the present embodiment, direction of pagemovement is along the lateral direction of drawing area 230 (FIG. 3)and, therefore, X coordinates of finger positions are compared asdescribed above.

If the determination at step 630 is positive, at step 632, a pagedisplayed in drawing area 230 is changed in accordance with thedirection of finger movement. Specifically, if the direction of fingermovement is to the left in FIG. 3, the next page of the current page isdisplayed on drawing area 230, and if it is to the right, the previouspage of the current page is displayed on drawing area 230. If thedetermination at step 630 is negative, at step 634, CPU 102 cancelsscrolling of the image displayed in drawing area 230. In the presentembodiment, if fingers are slid to the left/right in the state ofmulti-touch, the image scrolls to the left/right correspondingly.Therefore, if the length of sliding is equal to or smaller than thethreshold value D_(TH1), it is necessary that the screen image displaymust be returned to the original state. At step 634, the display ondrawing area 230 is returned from the scrolled image to the display oforiginal page. After the process of steps 632 and 634, the control flowreturns to step 600, to wait for detection of the next position oftouching.

If the determination at step 610 is negative, that is, if it isdetermined that all positions of multi-touch are maintained, at step612, CPU 102 determines whether or not finger positions detected at step608 are all in the left direction when viewed from the finger positionsrecorded at step 606. If the determination is positive, at step 614, CPU102 calculates the amount of movement (absolute value of difference in Xcoordinate values), and determines whether or not the value is largerthan a threshold value D_(TH2). In the present embodiment, the thresholdvalue D_(TH2) here is larger than the threshold value D_(TH1) at step630. Further, in the present embodiment, the amount of movement is anaverage value of all amounts of movement of multi-touch fingerpositions.

If the determination at step 614 is positive, at step 616, CPU 102 setsthe screen image to be displayed on drawing area 230 to be the image ofprevious one page. Specifically, if the current image is of the secondpage, the image of the first page is displayed on drawing area 230 bythe process of step 616. Thereafter, the control flow returns to step600. On the other hand, if the determination at step 614 is negative, atstep 618, CPU 102 scrolls the screen image displayed on drawing area 230to the left by the same length as the amount of movement of fingerpositions calculated at step 614. At this time, if there is a previouspage, the left end of the page is displayed at the right side of drawingarea 230. Then, the control flow proceeds to step 608.

If the determination at step 612 is negative, at step 620, CPU 102determines whether or not the finger positions detected at step 608 arein the right direction when viewed from the finger positions recorded atstep 606. If the determination is positive, at step 622, CPU 102calculates the amount of movement (absolute value of difference in Xcoordinate values), and determines whether or not the value is largerthan a threshold value D_(TH3). The threshold value here is the same asthe threshold value D_(TH2) at step 614.

If the determination at step 622 is positive, at step 624, CPU 102 setsthe screen image to be displayed on drawing area 230 to be the image ofnext one page. Specifically, if the current image is of the second page,the image of the third page is displayed on drawing area 230.Thereafter, the control flow returns to step 600. On the other hand, ifthe determination at step 622 is negative, CPU 102 scrolls the screenimage displayed on drawing area 230 to the right by the same length asthe amount of movement of finger positions calculated at step 622. Atthis time, if there is a next page, the right end of the page isdisplayed at the left side of drawing area 230. Then, the control flowproceeds to step 608.

If the determination at step 620 is negative, that is, if X coordinatesof finger positions are unchanged, the control proceeds to step 628. Atstep 628, CPU 102 executes a predetermined process in accordance withthe values of X and Y coordinates of finger positions or historythereof. The process that takes place at step 628 may include a processof updating display, when a specific button is pressed with multi-touch,when a specific plurality of buttons are pressed with multi-touch, whena so-called pinch-out is done with the space between fingers made wider,or a so-called pinch-in is done with the space between fingers madenarrower. At step 628, such a process is done. Since the contents ofprocessing at step 628 are not related to the present invention,detailed description thereof will not be given here, for simplicity ofdescription. After step 628, the control flow returns to step 608.

The control structure of the routine realizing the process formulti-touch operation is as described above.

If it is determined at step 604 that the detected finger position is oneand it is not a multi-touch operation, the process following step 636 isexecuted. At step 636, CPU 102 determines whether or not the finger hasmoved away from drawing area 230 of touch detecting unit 112.

If the determination at step 636 is positive, at step 638, based on thefinger position at the start of touching detected at step 600 and thefinger position immediately before detection of the finger moved away atstep 604, CPU 102 executes a process in accordance with these positions.By way of example, the process when function button 242, NEXT button 252or PREVIOUS button 254 shown in FIG. 3 is pressed corresponds to theprocess that is executed here. Thereafter, the control flow returns tostep 600, and the output of touch detecting unit 112 is monitored untilthe next touch is detected.

If the determination at step 636 is negative, it means that only onefinger position is continuously being detected. Here, at step 640, CPU102 determines whether or not the detected finger position is in drawingarea 230. If the determination at step 640 is positive, at step 642, CPU102 draws an image in accordance with drawing settings at that time onthe finger position. Thereafter, the control flow returns to step 602.If the determination at step 640 is negative, drawing is unnecessaryand, therefore, the control flow directly returns to step 602.

[Operation]

Electronic blackboard apparatus 100 operates in the following manner. Inthe following, the operation of electronic blackboard apparatus 100related mainly to the page feed will be described. Operations ofportions not specifically related to the present invention will not begiven here.

Referring to FIG. 9, assume that the third page of a document isdisplayed on the screen of display unit 114. Here, assume that a user680 touches the screen surface of display unit 114 with two fingers, andslides the two fingers to the right as represented by an arrow 682, withthe fingers kept in touch with the surface. The amount of sliding hereis a distance D₁, which is smaller than the threshold value D_(TH1).

Referring to FIG. 8, the control flow of the program at this time willbe described. At step 600, the first finger position is detected and,thereafter, the next finger position is detected at step 602. Here, theoperation is multi-touch operation and, therefore, the determination atstep 604 is YES. The first touched positions of two fingers are storedin RAM 106 at step 606. At step 608, current finger positions are againdetected. Assuming that the user continues sliding as shown in FIG. 9,the determination at step 610 is negative. Therefore, at step 612,whether or not the two finger positions are moving to the left isdetermined. Here, the two finger positions are both moving to the right.Therefore, the result of determination is negative. As a result, thecontrol proceeds to step 620. At step 620, whether or not the fingerpositions are moving to the right is determined. In the state shown inFIG. 9, the result of determination is positive. Therefore, next, atstep 622, whether or not the amount of movement of the fingers is largerthan the threshold value D_(TH1) is determined. At the beginning ofrepetition, the amount of movement is smaller than the threshold valuearm. Therefore, the result of determination is negative. As a result,the screen image is displayed scrolled to the right by the amount equalto the amount of movement of the fingers (average of the amounts ofmovement of two finger positions) at step 626. Thereafter, the controlreturns to step 608 and the next repetition process starts.

When the user continues sliding, the process described above is repeatedtime and again. As a result, if the distance D₁ of finger movement isequal to or smaller than the threshold value D_(TH1), the display ondisplay unit 114 is as shown in FIG. 10. Referring to FIG. 10, assumethat user 680 touches the display surface with his/her fingers andslides the two fingers by the distance D₁ (≤D_(TH1)) with the fingerskept in touch with the display surface. The screen image is scrolled tothe right by the same distance as D₁. As a result, on the left end ofthe screen image, a right end portion 696 of the image of the next pageis displayed.

Assume that after sliding his/her fingers by the distance D₁(<D_(TH1))to position 692, the user moves his/her fingers away from the displaysurface as shown by an arrow 690 and stops sliding. Here, the result ofdetermination at step 610 of FIG. 8 becomes positive, and the controlproceeds to step 630. Since D₁≤D_(TH1), the determination at step 630 isnegative, and scroll is canceled at step 634. As a result, the screenimage returns to the state shown in FIG. 9.

On the other hand, referring to FIG. 11, assume that after slidinghis/her fingers by the distance D₂ (D_(TH1)<D₂≤D_(TH2)) to position 716,user 680 moves his/her fingers away from the display surface as shown byan arrow 712 and stops sliding. Immediately before the fingers are movedaway, a right end portion 718 of the image of next page is displayed atthe left end of the screen image, as in the example of FIG. 10. Thewidth of right end portion 718 is substantially the same as distance D₂.When the fingers are moved away, here, the result of determination atstep 610 becomes positive and the control proceeds to step 630. As theresult of determination at step 630 is positive, at step 632, a processfor feeding the screen image by one page to the right is executed.Specifically, as shown by an arrow 720 in FIG. 11, the image of the nextpage is scrolled to the right by one page, and the next page isdisplayed on display unit 114 as shown in FIG. 12. In the example shownin FIGS. 11 and 12, the screen image before movement is the third page,and the image after movement is the fourth page.

Referring to FIG. 8, thereafter, the control proceeds to step 600 andCPU 102 executes the process of monitoring the output of touch detectingunit 112 until the user touches the screen image next time.

Here, assume that different from the example of FIG. 10, even aftersliding the fingers by the distance D₂, the user does not stop slidingbut further continues sliding to the right. Here, the result ofdetermination at step 610 is negative, and the control proceeds to steps612, 620 and 622. Referring to FIG. 13, while the slid distance D₃satisfies the relation D₃≤D_(TH3), the determination at step 622 isnegative. Therefore, at step 626, the screen image attains to the statein which the screen image is scrolled to the right by the distance D₃.In the example shown in FIG. 13, a right end portion 744 of the nextscreen image is displayed at the left end of the screen image. The widthof right end portion 744 is substantially the same as distance D₃.Thereafter, the control returns to step 608, and the next repetitionstarts.

Assume that, in this state, the user 680 further continues sliding tothe right and the distance D₃ becomes larger than the threshold valueD_(TH3). Here, the result of determination at step 622 becomes positive,and at step 624, the screen image is fed to the right by one page.Specifically, as shown in FIG. 14, the screen image of the fourth pageis displayed on display unit 114. Thereafter, the control returns fromstep 624 to step 600, and the next position of touching by the user isdetected. If the user 680 continues sliding of his/her fingers to theright as shown by an arrow 760, this sliding of the user is detected asa new touch at step 600. Therefore, through the process of steps 600,602 and 604, the operation described above is repeated, using theposition touched by the user 680 at the time point of page switching asa head position.

The description above relates to sliding by the user to the rightdirection. It is apparent from the description that similar operationtakes place when the sliding is to the left. Therefore, detaileddescription thereof will not be repeated.

When an upward/downward sliding, a pinch-out or a pinch-in takes placewith multi-touch, the control flows through steps 610, 612, 620 and 628of FIG. 8 while the sliding is being done, and the process in accordancewith the user operation is executed. If any operation is done with asingle-touch, the control proceeds from step 604 to step 636. Until thefinger is moved away, the control proceeds from step 640 to 642 if theoperation is in drawing area 230, and by the process at step 642, animage is drawn at the finger position. If the operation is out ofdrawing area 230, nothing happens in the present embodiment.

If the finger is moved away in the single-touch operation, the controlproceeds to steps 604, 636 and 638. At step 638, based on the firstdetected finger position and the finger position immediately before thefinger is moved away, a process in accordance with these positions isexecuted. If it is the case that the user touched any of functionbuttons 242 and moved away the finger, a predetermined processcorresponding to the touched function button 242 is executed at step638.

As described above, in electronic blackboard apparatus 100 in accordancewith the second embodiment, if the user slides his/her fingers in thelateral direction on the screen image with multi-touch, the operation isas follows. During sliding, the screen image is scrolled in accordancewith the amount of sliding. When the fingers are removed after slidingby the distance D₁ and the distance D₁ is equal to or smaller than thethreshold value D_(TH1), scroll is cancelled and the screen image beforesliding is resumed. If the distance D₁ is larger than the thresholdvalue D_(TH1), the screen image if moved by one page. In the secondembodiment described above, the next page is displayed when slid to theright, and the previous page is displayed when slid to the left.

If the user slides his/her fingers by a distance longer than thethreshold value D_(TH1), and the slid distance D₁ is equal to or smallerthan D_(TH3), page switch does not takes place, and the screen image isscrolled in accordance with the amount of sliding. If the distance D₁exceeds the threshold value D_(TH3), page switch automatically takesplace. If sliding is further continued, the same operation as whensliding is newly started is executed.

According to the second embodiment, if sliding is done with multi-touch,the screen image is scrolled. Therefore, it is possible for the user tointuitively understand that the screen image can be scrolled by amulti-touch operation. If touching fingers are removed in the course ofmulti-touch sliding and the amount of sliding is small, the scroll iscanceled and the original screen image is resumed. Even if multi-touchsliding operation is done erroneously during writing on the screenimage, the screen image returns to the normal state immediately when thefingers are moved away. Therefore, even when an erroneous operation isdone, the influence on the writing operation is small. On the otherhand, if the fingers are moved away after sliding over a certaindistance, switching to the next page (page feed) occurs. Therefore, thepage feed can be realized by an intuitive operation of multi-touchsliding operation. If sliding is further continued with the fingers kepttouched on the screen, page feed is automatically executed while slidingcontinues, and the sliding operation can further be continued.Therefore, advantageously, the operation of continuous page feed is madesimple.

In the second embodiment described above, it is determined at step 610of FIG. 8 that sliding ended if only one of multi-touching fingers ismoved away from the input screen image. The present invention, however,is not limited to such an embodiment. It may be determined that slidingended only when all fingers are moved away. In other words, if it is amulti-touch operation when sliding starts, sliding may be continuedthereafter even when the multi-touching is lost, and similar effects asin the embodiment above can be attained.

Further, in the second embodiment described above, determination as towhether an operation is a multi-touch operation or not is made at step604 of FIG. 8, and if it is determined not to be a multi-touchoperation, a process such as drawing, different from the page switchingor scroll process is executed. The present invention, however, is notlimited to such an embodiment. By way of example, the process of pageswitching or scroll may be executed only when the operation is amulti-touch with three or more fingers, and, if touched by one or twofingers, a process such as drawing, other than the page switching orscroll may be executed even if it is a multi-touch with two fingers.Generally speaking, the process of page switching or scroll may beexecuted only when the operation is a multi-touch with N fingers, and aprocess such as drawing, other than the page switching or scroll may beexecuted even if it is a multi-touch with N−1 or smaller number offingers, with N being an integer larger than 1.

Further, as to the method of calculating the distance D₁, use of theaverage value of amounts of movement of a plurality of fingers in thesecond embodiment is not limiting. By way of example, the amount ofmovement may be calculated based on only the first detected one fingerposition. Further, threshold values D_(TH1) and D_(TH2) mentioned abovemay be changed in accordance with the speed of movement of fingerpositions during sliding.

In the second embodiment described above, it is assumed that sliding andpage turning are along the lateral direction (X axis direction) of thescreen image. It is clear that the present invention is not limited tosuch an embodiment. For instance, the direction of sliding and pageturning may be the upward/downward direction (Y axis direction). Slidingmay be done in consideration of both X and Y axes directions. In otherwords, the present invention is applicable to an embodiment allowingsliding in a diagonal direction. Here, the first and second thresholdvalues may be determined separately for the X axis direction and Y axisdirection, or determined to be the same regardless of the direction.Further, the length of sliding may not be divided to components in the Xaxis direction and Y axis direction, and whether a page is to be turnedor not may be determined based on the length of the track of sliding(distance between the start point and end point).

Though an example of drawing and sliding with a finger or fingers hasbeen described in the second embodiment above, the present invention isnot limited to such an embodiment. For example, a dedicated pen may beused. As to the method of detecting a touched position on thetouch-panel, any method may be used provided that touching at aplurality of positions can be detected.

The embodiments as have been described here are mere examples and shouldnot be interpreted as restrictive. The scope of the present invention isdetermined by each of the claims with appropriate consideration of thewritten description of the embodiments and embraces modifications withinthe meaning of, and equivalent to, the languages in the claims.

What is claimed is:
 1. An image display apparatus allowing touch-input,comprising: a touch detecting unit that has a display screen, displayspage images page by page on the display screen, and detects positionsand a number of touch inputs that designate positions on said displayscreen; a scroll unit that scrolls, when a plurality of touch inputs aredetected by said touch detecting unit and their positions on the displayscreen move in one same direction, an image displayed on said displayscreen along with movement of positions of said plurality of touchinputs; and a first page switching unit that switches the image on saiddisplay screen by one page in accordance with the direction of movementof said plurality of touch inputs in response to an amount of movementof the touch inputs becomes larger than a first threshold value afterthe plurality of touch inputs are detected by said touch detecting unit;wherein the first page switching unit detects, after the plurality oftouch inputs are detected by said touch detecting unit, whether thenumber of touch inputs being decreased to zero, from an output of saidtouch detecting unit, and at a time when the number of touch inputs isdecreased to zero: if an amount of movement of the touch inputs is notlarger than a second threshold value smaller than the first thresholdvalue, executes a process of returning the image on said display screento a state before scrolling, and if an amount of movement of the touchinputs is equal to or greater than the second threshold value and equalto or smaller than the first threshold value, executes a process ofswitching the image by one page in accordance with the direction ofmovement of the plurality of touch inputs, and wherein after theplurality of touch inputs are detected by said touch detecting unit,when decrease in the number of touch inputs is detected from an outputof said touch detecting unit, even if an amount of movement of the touchinputs is not larger than the first threshold value, scrolling ismaintained as long as at least one touch input is maintained.
 2. Acontroller for a display apparatus, used for the display apparatus thathas a display screen, displays an image received from outside on thedisplay screen and detects and outputs, to the outside, positions and anumber of touch inputs that designate positions on said display screen,comprising: a scroll unit that generates, when it is detected that aplurality of touch inputs are detected by said display apparatus andtheir positions on the display screen move in one same direction, animage by scrolling the image displayed on said display screen along withthe movement of positions of said plurality of touch positions, andapplies the generated image to said display apparatus; a page switchingunit that switches the image on said display screen by one page inaccordance with the direction of movement of said plurality of touchinputs in response to an amount of movement of the touch inputs becomeslarger than a first threshold value after the plurality of touch inputsare detected by said display apparatus; the page switching unit detects,after the plurality of touch inputs are detected by said displayapparatus, whether the number of touch inputs being decreased to zero,from an output of said display apparatus, and at a time when the numberof touch inputs is decreased to zero: if an amount of movement of thetouch inputs is not larger than a second threshold value smaller thanthe first threshold value, executes a process of generating page imagedata for returning the image on said display screen to a state beforescrolling, and if an amount of movement of the touch inputs is equal toor greater than the second threshold value and equal to or smaller thanthe first threshold value, executes a process of switching the image byone page in accordance with the direction of movement of the pluralityof touch inputs, and transmits the generated page image data to saiddisplay apparatus, wherein after the plurality of touch inputs aredetected by said display apparatus, when decrease in the number of touchinputs is detected from an output of said display apparatus even if anamount of movement of the touch inputs is not larger than the firstthreshold value, scrolling is maintained as long as at least one touchinput is maintained.
 3. An image display apparatus allowing touch input,comprising: a touch detecting unit that has a display screen, displayspage images page by page on the display screen, and detects positionsand a number of touch inputs that designate positions on said displayscreen; a scroll unit that scrolls, when a plurality of touch inputs aredetected by said touch detecting unit and their positions on the displayscreen move in one same direction, an image displayed on said displayscreen along with movement of positions of said plurality of touchinputs; a first page switching unit that switches the image on saiddisplay screen by one page in accordance with the direction of movementof said plurality of touch inputs in response to an amount of movementof the touch inputs becomes larger than a first threshold value afterthe plurality of touch inputs are detected by said touch detecting unit;a returning unit that detects, from an output of said touch detectingunit, whether the number of said plurality of touch inputs beingdecreased to zero, and at a time when the number of touch inputs isdecreased to zero: if an amount of movement of the touch inputs is notlarger than a second threshold value smaller than the first thresholdvalue, returns scrolling of the image by said scroll unit to a statebefore scrolling, and if an amount of movement of the touch inputs isequal to or greater than the second threshold value and equal to orsmaller than the first threshold value, executes a process of switchingthe image by one page in accordance with the direction of movement ofthe plurality of touch inputs, wherein after the plurality of touchinputs are detected by said touch detecting unit, when decrease in thenumber of touch inputs is detected from an output of said touchdetecting unit and, even if an amount of movement of the touch inputs isnot larger than the first threshold value, scrolling is maintained aslong as at least one touch input is maintained.
 4. A controller for adisplay apparatus, used for the display apparatus that has a displayscreen, displays an image received from outside on the display screenand detects and outputs, to the outside, positions and a number of touchinputs that designate a position on said display screen, comprising: ascroll unit that generates, when it is detected that a plurality oftouch inputs are detected by said display apparatus and their positionson the display screen move in one same direction, an image by scrollingthe image displayed on said display screen along with the movement ofpositions of said plurality of touch positions, and applies thegenerated image to said display apparatus; a first page switching unitthat switches the image on said display screen by one page in accordancewith the direction of movement of said plurality of touch inputs inresponse to an amount of movement of the touch inputs becomes largerthan a first threshold value after the plurality of touch inputs aredetected by said display apparatus; and a returning unit that detects,from an output of said display apparatus, whether the number of saidplurality of touch inputs being decreased to zero, and at a time whenthe number of touch inputs is decreased to zero: if an amount ofmovement of the touch inputs is not larger than a second threshold valuesmaller than the first threshold value, returns scrolling of the imageby said scroll unit to a state before scrolling, and if an amount ofmovement of the touch inputs is equal to or greater than the secondthreshold value and equal to or smaller than the first threshold value,executes a process of switching the image by one page in accordance withthe direction of movement of the plurality of touch inputs, whereinafter the plurality of touch inputs are detected by said displayapparatus, when decrease in the number of touch inputs is detected froman output of said display apparatus and, even if an amount of movementof the touch inputs is not larger than the first threshold value,scrolling is maintained as long as at least one touch input ismaintained.
 5. A display control method of an image display apparatusthat allows touch-input, comprising: displaying page images page by pageon a display screen, and detecting positions and a number of touchinputs that designate positions on said display screen; scrolling, whena plurality of touch inputs are detected and their positions on thedisplay screen move in one same direction, an image displayed on saiddisplay screen along with movement of positions of said plurality oftouch inputs; after the plurality of touch inputs are detected, inresponse to an amount of movement of the touch inputs becomes largerthan a first threshold value, switching the image on said display screenby one page in accordance with the direction of movement of saidplurality of touch inputs; and detecting, after the plurality of touchinputs are detected, whether the number of touch inputs being decreasedto zero, and at a time when the number of touch inputs is decreased tozero: if an amount of movement of the touch inputs is not larger than asecond threshold value smaller than the first threshold value, executesa process of returning the image on said display screen to a statebefore scrolling, and if an amount of movement of the touch inputs isequal to or greater than the second threshold value and equal to orsmaller than the first threshold value, executes a process of switchingthe image by one page in accordance with the direction of movement ofthe plurality of touch inputs, wherein when a decrease in the number oftouch inputs is detected after the plurality of touch inputs aredetected, even if an amount of movement of the touch inputs is notlarger than the first threshold value, scrolling is maintained as longas at least one touch input is maintained.
 6. A display control methodof an image display apparatus that allows touch-input, comprising:displaying page images page by page on a display screen, and detectingpositions and a number of touch inputs that designate positions on saiddisplay screen; scrolling, when a plurality of touch inputs are detectedand their positions on the display screen move in one same direction, animage displayed on said display screen along with movement of positionsof said plurality of touch inputs; after the plurality of touch inputsare detected, in response to an amount of movement of the touch inputsbecomes larger than a first threshold value, switching the image on saiddisplay screen by one page in accordance with the direction of movementof said plurality of touch inputs; detecting, from the detectedplurality of touch inputs, whether the number of said plurality of touchinputs being decreased to zero, and at a time when the number of touchinputs is decreased to zero: if an amount of movement of the touchinputs not larger than a second threshold value smaller than the firstthreshold value, returning scrolling of the image to a state beforescrolling, and if an amount of movement of the touch inputs is equal toor greater than the second threshold value and equal to or smaller thanthe first threshold value, executes a process of switching the image byone page in accordance with the direction of movement of the pluralityof touch inputs, wherein when a decrease in the number of touch inputsis detected after the plurality of touch inputs are detected, even if anamount of movement of the touch inputs is not larger than the firstthreshold value, scrolling is maintained as long as at least one touchinput is maintained.